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ABSTRACT 


This  thesis  explores  a method  of  aggregating  the 
measures  of  effectiveness  of  a weapon  system  from  its 
characteristics.  With  this  method,  the  constant  sum  method 
and  multiple  regression  are  used  to  develop  a functional 
relationship  between  system  effectiveness  and  system 
character istics . As  an  example,  a study  of  a tank  weapon 
system  was  conducted  with  data  from  the  U.S.  Army  Armor 
School.  It  was  concluded  that  the  aggregation  method  is 
feasible,  and  that  for  the  tank  system  studied,  the 
reciprocals  of  system  character istics  give  a good  estimating 


equation  for  measuring  tank  system  effectiveness. 
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I . INTRODUCTION 


We  often  find  measurement  problems  in  Operations 
Research  that  are  difficult  in  that  widely  used  concepts 
have  not  been  made  operational.  How  to  measure  the  effec- 
tiveness of  a weapon  system  is  one  of  the  most  important 
tasks  in  military  affairs.  What  is  needed  is  some  method 
to  give  answers  to  such  questions  as  "Which  tank  is  more 
effective?",  or  "How  much  better  a M60A1  is  than  a T-62?" 

We  want  to  know  how  much  better  one  weapon  system  is 
than  another  among  similar  systems.  One  way  of  doing  this 
is  through  a functional  relationship  between  system  effec- 
tiveness and  the  characteristics  of  the  system. 

In  this  paper  we  will  propose  and  demonstrate  a way  of 
structuring  such  a relationship  using  as  values  for  the 
effectiveness  of  weapon  systems  values  which  originally 
came  from  military  experts'  judgements.  Once  we  have  found 
such  a function,  we  would  not  necessarily  require  experts 
judgements  again,  since  one  can  use  the  function  to  calcu- 
late the  effectiveness  of  a proposed  weapon  system  from  its 
characteristics . 

Chapter  II  gives  the  concept  and  general  procedure  of 
this  approach  to  measuring  effectiveness.  For  an  illus- 
trative problem  we  will  discuss  the  selection  of  the 
systems  and  their  major  characteristics,  the  preparation  of 
questionnaires  and  the  selection  of  "expert"  judges. 
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Using  information  from  the  judges,  we  will  show  in 
Chapter  II I how  to  compute  the  values  of  effectiveness  of 


weapon  systems  using  the  constant  sum  method.  In  seeking 
a good  functional  relationship  between  systems  effective- 
ness and  system  characteristics , we  will  use  an  APL  computer 
program  for  multiple  regression  and  explore  various  func- 
tional forms  by  transforming  the  data.  This  is  the  content 
of  Chapter  IV.  In  the  final  chapter,  we  will  summarize 
major  conclusions  and  observations  during  the  course  of 
this  research. 

It  must  be  emphasized  that  although  the  illustration 
study  presented  here  involves  the  effectiveness  of  a main 
battle  tank,  our  purpose  is  to  demonstrate  a proposed 
approach  to  the  measurement  of  systems  effectiveness,  and 
not  to  develop  effectiveness  relationships  for  main  battle 
tanks.  A study  of  the  latter  type  would  require  resources 
in  excess  of  those  available  for  this  work. 


DESIGN  AND  DATA  COLLECTION 


II . 

This  chapter  describes  the  proposed  method  or  finding  a 
useful  functional  relationship  between  weapon  system 
characteristics  and  the  overall  effectiveness  of  a system, 
and  in  particular  describes  and  demonstrates  the  collection 
of  data  needed  for  this  approach.  First  we  will  give  the 
concept  of  the  method  and  the  general  procedure  we  are  ocmg 
to  follow.  Then,  using  tanks  as  an  example,  we  will  discuss 
the  selection  of  systems  and  their  major  characteristics, 
and  comment  about  preparing  questionnaires . 

A.  CONCEPT  OF  THE  PROBLEM 

The  problem  we  are  interested  in  is  to  calculate  the 
overall  effectiveness  of  a weapon  system  from  its  charac- 
teristics. The  approach  here  will  be  to  demonstrate  how 
to  estimate  the  effectiveness  of  a weapon  system  using  a 
scaling  method  [1], 

Every  weapon  system  has  its  own  characteristics  and  if 
we  have  a value  for  the  overall  effectiveness  of  that  system, 
we  should  be  able  to  obtain  or  fit  a relationship  expressing: 

Overall  effectiveness  = - x X X ' ' ) 

of  a weapon  system  ~ 1"  2'  ’’’’  m 

where 


X . , X., , ...,  X are  svstem  character  ist  ics  . 

1 - m 
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The  purpose  of  this  paper  is  to  show  a procedure  for 
obtaining  values  for  overall  effectiveness,  and  a way  of 
determining  the  tunc* • on  f.  Since  we  will  be  fitting  func- 
tions to  data,  the  more  instances  of  the  system  we  use,  the 
better  the  functional  relationship  we  can  find. 

Figure  1 shows  the  relationship  between  characteristics 
and  system  ef fectiveness . 


SYSTEM  CHARACTERISTICS 


INSTANCE 

EFFECTIVENESS 
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FIG.  1 RELATIONSHIP  BETWEEN  EFFECTIVENESS 
AND  CHARACTERISTICS 


There  are  n instances  of  the  system  and 
values  which  have  to  be  obtained.  The 
acteristics  which  we  presume  to  relate 
and  assume  m n.  If  we  express  this  f 
equation  we  would  write: 


thus  n effectiveness 
system  has  m char- 
to  effectiveness, 
igure  in  a mathematical 


= f(xu,xi2, 


X . ) 

im 


(2) 
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describing  overall  effectiveness  of  a system  as  a function 
of  its  charact' ristics , and  we  propose  to  show  how  to 


obtain  values  for  the  Y^,  and  how  to  find  a good  fitting 


function  f. 


B.  GENERAL  STUDY  PROCEDURE 


The  general  study  procedure  is  composed  of  3 steps. 

Step  1.  Design  of  the  study.  In  this  step  we  have  to 
select  the  instances  of  the  system  and  the  major  charac- 
teristics of  that  system,  and  collect  the  data  to  provide 
characteristic  values  for  each  selected  instance.  A scaling 
method  must  then  be  chosen  by  which  we  can  quantify  information 
from  judges  about  system  effectiveness.  After  doing  this, 
we  prepare  questionnaires  and  send  them  to  selected  judges 
to  obtain  information  from  them  about  the  effectiveness 
of  the  system. 

Step  2.  Computation  of  the  effectiveness  of  each 
instance  of  the  system.  The  questionniares  sent  to  the 
judges  are  to  obtain  expert  judgement,  which  may  be  expressed 
as  a ranking  of  systems  or  as  a ratio  scale  value  of  effec- 
tiveness, according  to  the  requirement  of  the  scaling  method 
which  has  been  selected.  Based  on  this  information,  the 
next  thing  to  do  is  to  use  the  scaling  method  to  compute 
the  effectiveness  of  each  instance  of  the  system. 

Step  3.  Determination  of  the  functional  relationship. 

Once  we  have  the  numberical  values  of  effectiveness  and  the 


characteristics  for  each  system,  the  next  step  is  similar  to 
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PROBLEM  OBJECTIVE 
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finding  a cost  estimating  equation  [2].  Because  of  many 
characteristics  and  instances  of  the  system,  it  seems 
reasonable  to  use  the  multiple  regression  technique  to 
find  a good  estimating  equation.  Since  we  will  not  restrict 
ourselves  to  the  linear  equation  case,  there  are  many 
candidate  functions  available  by  transforming  the  data. 
Therefore  we  have  to  choose  the  best  functional  relation- 
ship by  evaluating  these  candidate  equations.  This  whole 
procedure  is  represented  in  Figu.e  2. 

C.  DETAILED  PROCEDURE 

We  will  discuss  the  elements  of  the  design  of  the  study 
in  detail  in  this  section.  Steps  2 and  3 (computation  of 
effectiveness  and  determination  of  a functional  relationship) 
will  be  discussed  in  detail  in  Chapters  III  and  IV. 

1.  Selection  of  the  Competing  Systems 

Among  various  types  of  weapon  systems  we  could  use 
as  the  example  in  this  paper,  let's  consider  the  tank 
weapon  system.  What  kind  of  tank  should  we  choose?  It 
depends  on  what  we  are  going  to  do  with  the  tanks,  and  since 
we  want  to  decide  which  tank  system  is  good  for  battle,  we 
shall  choose  Main  Battle  Tanks  (MBTs)  [3]. 

Wi  shall  define  Main  Battle  Tanks  (MBTs)  as  the 
greatest  number  of  tanks  a country  has  for  battle.  For 
this  study  six  Main  Battle  Tanks  were  chosen: 
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M4  8A5 
M60A1 
AMX-30 
LEOPARD  1 
T-b2 

and 

CENTURION  (Mk .13) 

The  reason  why  these  tanks  are  chosen  is  that  (1)  these 
are  all  Main  Battle  Tanks  that  are  currently  in  service,  and 
(2)  they  are  very  well-known  tanks  so  that  the  experts  can 
give  knowledgeable  information.  If  some  of  the  tanks  were 
older  ones  not  in  service  or  tanks  under  development,  some 
judges  might  not  be  familiar  with  them. 

As  mentioned  before,  the  more  tanks  used  in  the 
study,  the  better  is  the  relationship  that  may  be  obtained. 

The  reason  for  using  only  six  MBTx  here  relates  to  the  work- 
load that  is  placed  on  the  judges.  In  the  constant  sum 
scaling  method  which  we  are  going  to  use,  each  judge  is 
asked  to  consider  each  possible  pair  of  instances  and  split 
100  points  between  the  two  instances  in  each  pair.  Thus  for 
n instances  to  be  scaled,  n(n-l)/2  pairs  must  be  considered 
by  each  judge.  Therefore  we  have  to  consider  the  number  of 
instances  to  be  scaled  because  the  greater  the  number  of 
instances,  the  greater  the  workload  asked  of  each  judge 
and  the  less  careful  he  might  be  in  his  rating  and  the  smaller 
proportion  of  questionnaires  we  would  expect  to  be  returned. 


(Korea) , 

( U . S . A . ) , 
(France) , 

(W.  Germany), 
(U.S.S.R. ), 

(Great  Britain). 


i 


i 

. 


I 
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Since  "the  more  judges  the  better"  is  particularly  an  axiom 
in  scaling,  tradeoffs  may  have  to  be  made  between  the  effort 
that  will  be  required  of  a judge  and  the  amount  of  confidence 
one  wishes  to  have  in  the  resulting  scale.  Our  selection 
of  six  tanks  (instances)  requires  6 = 15  pairs  of  tanks 

to  be  considered,  and  this  number  is  thought  reasonable  for 
a judge  to  handle  in  a short  period  of  time. 

2 .  Selection  of  the  Major  Characteristics 

There  are  of  course  many  character istics  which 
affect  the  effectiveness  of  a system.  For  example,  one 
source  lists  about  30  characteristics  which  are  deemed 
relevant  to  the  importance  of  a tank  system  [3]. 

Some  of  the  characteristics  have  very  similar  values 
among  various  MBTs , and  these  character istics  should  not 
make  any  substantial  difference  in  the  comparison  of  effec- 
tiveness among  the  competing  systems.  Thus  we  don't  need  to 
consider  this  kind  of  characteristic. 

We  have  chosen  4 tank  characteristics  as  follows: 

1.  speed 

2.  silhouette 

3.  Hp/ton 

4.  armor. 

These  characteristics  are  not  necessarily  the  most  important 
ones.  For  example,  fire  power  is  a very  important  considera- 
tion, but  obtaining  useful  numerical  data  on  fire  power  is 
very  difficult  due  to  a lack  of  a standard  measurement  criterion. 
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For  fire  power,  we  could  consider  the  main  gun,  but  all  the 
selected  Main  Battle  Tanks  have  a similar  main  gun  with 
caliber  105  mm  except  T-62  which  has  115  mm  main  gun. 

Therefore  we  have  chosen  the  above  characteristics  as 
generally  accepted  important  factors,  which  should  serve 
well  in  our  demonstration  of  a method  for  assessing  system 
effectiveness . 

3 . Collection  of  Data 

After  selecting  the  system  instances  and  the  ma^or 
characteristics  for  the  study,  we  have  to  collect  data  for 
each  characteristic.  Table  I shows  the  basic  data  |3]  of  the 
six  Main  Battle  Tanks,  whLch  we  will  use.  Clearly,  one  should 
use  all  the  information  sources  available:  manufacturer, 
military  sources,  or  technical  reports. 


Table  I.  CHARACTERISTICS  OF  SIX  MAIN  BATTLE  TANKS  [3] 


CHARACTERISTICS 


SYSTEMS 

(mm  on  nose) 

(height  in  n) 

(km,  hr) 

Tj 

rr 

G 

3 

LEOPARD  1 

70 

2 . o4 

6 5 

20 . 7 

MbOAl 

110 

3.26 

43.3 

15.  3 

T-62 

100 

2.4 

50 

19.2 

M4  8A5 

110 

3 . 03 

43.2 

15.9 

CENTURION 
(Mk. 13) 

113 

3.01 

3 4.6 

12 . 5 

AMX-30 

43 

2.85 

65 

19.4 
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Select  ion  of  the  Method 


4 . 

There  are  many  scaling  methods  we  could  use  for  our 
study  such  as  a numerical  method,  the  constant  sum  method, 
a comparative  method,  or  a categorical  method.  For  our 
study  we  want  to  know  how  much  better  one  system  is  than 
another.  In  other  words,  we  want  a ratio  scale  that  can  be 
used  for  directly  comparing  two  systems. 

Among  those  sea  lino  methods  the  constant,  sum  method 
will  qive  a ratio  scale  which  is  easy  to  use.  We  can  convert 
the  information  from  judues  about  system  effectiveness  into 
a ratio  scale.  Therefore  the  constant  sum  method  was  chosen 
for  the  illustrative  study  of  tank  effectiveness. 

5 . Preparation  of  the  Questionnaire 

This  is  one  of  the  most  important  parts  of  the 
analysis.  The  quest lonnaire  should  be  prepared  very  care- 
fully with  a clear  explanation  of  how  to  fill  it  out, 
together  with  information  about  the  systems  which  the  judaes 
can  use  to  assist  them  in  their  ratings,  Since  we  are  going 
to  use  the  constant  sum  method  to  compute  the  effectiveness, 
the  fudges  will  be  asked  to  make  ratio  scale  judgements  by 
splitting  100  points  between  the  two  instances  represented 
by  each  possible  pair  of  n tanks.1  For  example  a judge 


Alternately,  of  course,  one  could  gather  information  from 
a judge  by  asking  for  a numerical  effectiveness  ratine  for  each 
tank,  so  that  a judge  would  assign  a number  to  each  tank 
reflecting  how  effective  he  thought  it  was.  This  process,  however 
is  a very  difficult  rating  task  for  the  judge.  Literature  m 
psychometrics  suggests  that  judges  will  reflect  their  feel  inns 
more  accurately,  and  with  more  confidence,  if  they  are  given  just 
two  instances  at  a time  to  rate,  as  in  the  constant  sum  method  [11 
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might  split  the  LOO  points  as 


M60A1  SO  T-62  20 

if  he  thinks  M60A1  is  much  better  (4  times  in  this  example) 
than  the  T-62  or  he  might  split  the  100  points  as 

M60A1  5jJ  T-62  50 

if  he  thinks  they  are  equal  in  terns  of  combat  effectiveness. 

For  our  study  we  have  six  instances,  and  therefore 
there  are  15  pairs  to  be  presented  to  the  judges  (the  ques- 
tionnaire which  was  used  is  shown  in  Appendix  A) . 

The  questionnaire  actually  used  consisted  of  two 
parts,  seeking  ratings  on  both  overall  tank  effectiveness 
and  on  the  contribution  to  tank  effectiveness  of  various 
character istics . Scaled  values  from  the  second,  supplemen- 
tary study  are  given  in  Appendix  C and  may  be  of  interest 
to  readers. 

6 . Selection  of  the  Judges 

There  appears  to  be  no  rule  or  standard  for  desig- 
nating individuals  as  "experts".  It  depends  on  common  sense 
or  military  judgements. 

We  believe  that  the  armor  officers  in  the  U.S.  Army 
Armor  school  may  be  considered  experts  about  tanks.  There- 
fore we  selected  them  as  judges  and,  after  obtaining  the 
gracious  cooperation  of  the  Armor  School,  sent  50  question- 
naires to  them.  The  50  questionnaires  were  completed  by 
all  ranks  ranging  from  Lieutenant  to  flag  officer,  and  all 
50  questionnaires  were  returned,  ready  for  analysis. 
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bo  tar  in  this  paper  we  have  structured  the  approach, 
selected  the  systems  we  are  interested  in  evaluating, 
collected  data  on  those  systems,  selected  an  analytic 
method  and  prepared  the  questionnaires  for  the  ]udges. 
in  the  next  chapter  we  win  describe  how  the  overall 
effectiveness  values  were  computed  from  the  information 
Provided  by  the  judcres. 


19 


III.  COMPUTATION  OF  SYSTEM  EFFECTIVENESS 


In  the  previous  chapter  we  showed  how  the  model  was 
organized,  and  how  the  questionnaires  were  prepared  for  the 
judges.  In  this  chapter  we  will  explain  the  constant  sum 
scaling  method  of  computing  the  overall  effectiveness  values 
for  each  system  instance,  using  the  information  received 
from  the  judges. 

A.  CALCULATION  OF  THE  EFFECTIVENESS  OF  EACH  WEAPON  SYSTEM 

USING  THE  CONSTANT  SUM  SCALING  METHOD 

The  constant  sum  scaling  method  is  designed  to  scale  a 
property  with  a natural  origin  or  an  origin  upon  which  the 
judges  agree.  Judges  are  asked  to  allocate  100  points 
between  two  instances,  considering  each  possible  pair  of 
instances.  Thus  the  resulting  scale  for  instances  of  the 
property  is  taken  as  a ratio  scale.  In  the  following  para- 
graphs we  will  explain  how  the  scale  values  are  obtained. 

Let  a^  be  the  number  of  points  out  of  100  which  a 
judge  awards  to  instance  j when  it  is  compared  to  instance 
i.  If  we  arrange  a judge's  responses  in  an  array  a^ ^ , 
there  will  be  one  array  for  each  judge  in  which  values  on 
the  diagonal  would  be  set  at  50  because  comparing  something 
with  itself  should  be  50:50. 

We  could  average  these  arrays  over  the  N judges  to 
produce  an  array  where 
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Mi 


*; — _ over  all  judges 

aij  a n — <3) 

and  the  values  on  the  diagonal  remain  set  at  50.  Table  III 
shows  the  a^  values  for  this  study,  as  computed  from  the 
data  collected  from  50  judges  (the  50  a^  arrays  of  individual 
judges  are  omitted) . 

From  Table  III  one  may  see,  for  example,  that  the 


average  of  the  judges' 

100-pomt  split 

in  overall 

tank 

effectiveness  between 

the  T-62  and  the 

AMX-30  was 

55.9  for 

the  T-62  and 

44.1  for 

the  AMX-30. 

Table  III 

. a . . Array  Computed  from 

Judces ' Re 

sponses 

- J 

M4  8A5 

M60A1  AMX-30 

LEOPARD 

1 T-62 

CENTER  I O’ 
(Mk. 13) 

M48A5 

50 

56.94  45.1 

64 . 22 

54.7 

45 . 4 

M60A1 

43.06 

50  39.14 

54 . 44 

45.04 

39.36 

AMX-30 

54.9 

60.36  50 

63.36 

55.9 

49  . ~ 

LEOPARD  1 

35.  *8 

45.56  36.14 

50 

41.46 

3~.  54 

T-62 

45.3 

54.96  44.1 

58.54 

50 

44 

CENTURION 
t.Mk.  13) 

54.6 

60 . 6 4 50.3 

62.46 

56 

50 

The  next 

step  is  t 

o construct  a new 

Wu  arrav 

where  the 

entries  are  the  ratios 

of  the  instance 

values  across  the 

diagonal,  or 

■ 

— 


(4) 


Of  course  the  diagonal  entries  in  the  array  should  be 

1.0.  In  this  array,  it  is  immediately  apparent  that  the 
entry  in  the  ith  row  and  jth  column  is  the  reciprocal  of 
that  in  the  jth  row,  and  ith  column,  i.e., 


Values  of  for  the  tank  data  were  computed  from  the 

data  in  Table  III,  and  are  shown  in  Table  IV. 


Table  IV.  VT„  Array 


M4SA5 

M60A1 

AMX-30 

LEOPARD  1 

T-62 

CENTURION 
(Mk. 13) 

M48A5 

1.322 

.321 

1.735 

1.20' 

.331 

M60A1 

. 756 

1 

. 643 

1.195 

. S19 

.649 

AMX-30 

1.217 

1.555 

1 

1.767 

1.267 

.933 

LEOPARD  1 

.557 

.337 

. 566 

1 

. 703 

.601 

T-62 

.323 

1.220 

.789 

1.412 

1 

. 'S6 

CENTURION 

1.203 

1.541 

1.012 

1.664 

1.273 

1 

(Mk.  13) 


Since  W . is  the  ratio  of  the  averaae  points  awarded  to 
L j 

3 l when  compared  to  i)  to  the  average  points  awarded  to  i 
(when  compared  to  j),  then  in  general,  if  and  are  the 


scale  values  we  seek,  is  an  estimate  of  the  ratio 

Sj/Si‘  T^us  terms  of  the  Table  IV  data,  for  example, 

judges  have  indicated  that  they  feel  that  the  M60A1  tank 
is  1.322  times  more  effective  than  the  M43A5  tank.  The 
solution  is  overdetermined,  however,  since  there  are  far 
more  ratios  (fifteen)  than  there  are  scale  values  to 

be  estimated  (six).  For  example,  one  could  also  compare 
the  M60A1  and  the  M48A5  by  comparing  both  against  the 
Leopard  1,  and  in  this  case  the  M60A1  would  be  judged 


0.837 

0.557 


1.503 


times  better  than  the  M43A5. 

We  propose  to  handle  this  multiple  estimate  problem 
by  a least  squares  approach  over  the  estimates. 

We  could  write 


Wij 


(6) 


by  taking  the  log  of  both  sides  of  (6) , we  have 


log  Sj  - log  Si  = log  W^  , 


or 


log  W^ j - (log  Sj  - log  S^)  = 0 . (7) 

For  the  least-squares  approach  we  wish  to  obtain  values  of 
S;  and  which  will  make 
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If  we  take  the  antilogarithms  of 

n 

l log  wij 

i=l 


equation  (10)  becomes 


n 

SJ  = n (Wij)1/n  , j = 1,2 n (11) 

i=l 

Therefore  the  scale  value  of  instance  j , S ^ , derived  from 
the  least  square  approach  can  be  interpreted  simply  as  a 
Geometric  mean  of  the  jth  column  of  the  W. ^ array.  Using 
Table  IV  (W^  array)  and  equation  (11) , we  obtained 
values  as  follows: 

SL  = 0.89 

52  = 1.22 

53  = 0.78 

54  = 1.44 

55  = 1.02 

Sg  = 0.79 

These  ratio  scale  values  for  the  overall  effectiveness  of 
six  tank  weapon  systems  will  be  discussed  in  the  next 
section . 
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B.  SYSTEM  EFFECTIVENESS  VALUES 

At  last  we  have  ratio  scale  values  tor  the  relative 
effectiveness  of  each  tank  as  shown  in  Table  V. 


Table  V.  EFFECTIVENESS  OF  SIX  MAIN  BATTLE  TANKS 


TANK 
M4  8A5 
MS0A1 
AMX- 30 
LEOPARD  1 
T-61 

CENTURION 
vMk.  13' 


EFFECTIVENESS 
0 . SO 
1.22 

o.  :s 

1.44 

1.0  2 
0 . 70 


If  we  represent  these  effectiveness  values  graphically, 
would  appear  as  in  Figure  3. 


LEOPARD  I 

MbOAl 

T-n2 
M4SA5 
CENTURION 
AMX- 30 


Fid.  3 EFFECTIVENESS  OF  SIX  MAIN  BATTLE  TANKS 


they 


. ..  . 


The  effectiveness  values  calculated  above  and  shown  here 
are  relative  effectiveness  values,  and  have  no  physical 
units . 

An  advantage  of  using  a scaling  method  which  requires 
that  the  judges  provide  ratio  scale  information  is  that  the 
output  is  also  a ratio  scale.  This  is  very  advantageous 
in  interpreting  the  results  in  that,  for  example,  we  can 
say  the  Leopard  1 is  more  effective  (or  better)  than  the 
AMX-30  by 


1.44  - 0.73 
1.44 


x 100 


46%  . 


Such  a statement  would  not  be  possible  if  the  scale  was 
interval  or  less.  The  effectiveness  value  by  itself  is 
meaningless,  but  because  this  is  a ratio  scale,  we  can 
compare  the  two  systems  directly  by  ratios,  and  can  say 
how  much  better  one  is  than  the  other. 

In  this  chapter  we  have  computed  the  effectiveness 
of  six  tank  weapon  systems  using  the  constant  sum  scaling 
method.  This  scaling  approach  provides  an  effective  way  of 
computing  the  overall  effectiveness  of  weapon  systems. 
However,  this  use  of  a scaling  method  alone  requires  that 
we  have  to  send  questionnaires  everytime  we  want  to  calcu- 
late the  effectiveness  of,  say,  a new  or  different  Main 
Battle  Tank.  This  is  because  the  effectiveness  we  computed 
is  based  on  the  information  given  by  the  judges,  but  not 
directly  on  the  system  characteristics. 
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Ir.  the  next  chapter,  we  are  going  to  find  a functional 
relationship  between  the  effectiveness  computed  from  the 
information  given  by  the  judges  and  the  characteristics  of 
systems.  Such  a relationship  could  be  used  in  subsequent 
analyses  instead  of  sending  questionnaires. 
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IV.  DETERMINATION  OF  A FUNCTIONAL  RELATIONSHIP 
FOR  SYSTEM  EFFECTIVENESS 


In  the  previous  chapter  we  computed  the  effectiveness 
of  six  Main  Battle  Tanks,  and  we  have  the  characteristics 
of  these  systems  from  Chapter  II.  In  this  chapter,  we 
are  going  to  find  a function  which  relates  the  character- 
istics to  the  effectiveness  values.  With  the  data  shown 
in  Table  I (characteristics ) , tank  characteristics  can  be 
thought  of  as  explantory  variables  and  the  effectiveness 
can  be  thought  of  as  a dependent  variable  for  the  multiple 
regression  [2]  analysis  which  will  now  be  discussed. 

A.  SEARCHING  FOR  A FUNCTIONAL  RELATIONSHIP  BETWEEN 
EFFECTIVENESS  AND  SYSTEM  CHARACTERISTICS  USING 
MULTIPLE  REGRESSION 

Since  there  are  six  dependent  variables  (six  Main 
Battle  Tanks)  and  four  explanatory  variables  (four  character- 
istics), computational  work  [4]  is  simplified  with  a 
computer  program  for  linear  and  non-linear  multiple 
regression.  There  exists  a very  useful  APL  computer 
program  for  multiple  regression  named  "REGRESS", “ which 
will  be  used  to  find  a good  functional  relationship  between 
effectiveness  and  system  characteristics . 

The  "REGRESS"  program  does  a multiple  regression  analysis 
relating  the  dependent  variables  Y to  a set  of  explanatory 


“See  Appendix  B. 


30 


variables  X.  Here  V is  a vector  of  size  n and  the  right 
hand  argument  X is  an  nxm  matrix  consisting  of  n observa- 
tions on  each  of  m variables,  corresponding  to  the  tank 
characteristic  data  in  Table  I. 

Output  consists  of  an  ANOVA  table,  coefficient  of 
2 

determination  R , Standard  Error  SE,  regression  coefficients 
bj  (the  first  coefficient  is  the  constant  term,  a) , and  a 
vector  of  predicted  Y values  and  residuals. 

For  the  analysis,  we  used  the  computer  program  "REGRESS" 
on  APL  by  taking 

(L)  a linear  combination  of  the  characteristics,  or 

m 

Ys  = a + l b.  X.  . , 

1 L 3 ij 

j = l 

(2)  a linear  combination  of  logs  of  the  characteristics , or 

m 

\ = a + l bj  log  Xij  , 

j-1 

(3)  a linear  combination  of  logs  of  both  the  characteristics 
and  effectiveness,  or 


m 

Yj_  = exp  [a  + l bj  log  X^]  , 

j-1 


31 


50  questionnaires  were  returned,  ready  for  analysis. 
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(4)  a linear  combination  of  reciprocal  of  the  character- 
istics, or 


m 


ri  = a + l bj  (3T-)  ' 
j=l  13 


(5)  a linear  combination  of  reciprocal  of  both  the 
characteristics  and  the  effectiveness,  or 


m 


'■i  = ta  + l b (rji-) 

3-1  : ij 


1 , ,-l 


(6)  a linear  combination  of  square  root  of  the 
characteristics,  or 


!i  ■ * * l b3  'xij  - 

1=1 


(7)  a linear  combination  of  reciprocal  of  square  root 
of  the  characteristics,  or 


Yi  = 3 + 


. . J /X.  . 

:=i  i: 


and  finally 
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lS)  a linear  combination  of  square  root  of  both  the 


characteristics  and  effectiveness,  or 


m 

Y ■ » [a  + y b . .X  . ] “ . 
i L : 13 

5-1 

From  the  computer  output  isee  Appendix  B) , the  results 
were  obtained  as  shown  in  Table  VI.  Here  R”  , the  coefficient 
of  determination , shows  the  proportion  of  total  variance 
accounted  for  by  the  estimating  equation  as  a measure  of 
dispersion,  and  thus  a bigger  R'  is  better.  The  third 
column  shows  the  standard  error  which  is  defined  as  the 
square  root  of  the  unexplained  variance  of  the  dependent 
variables  Y.  Therefore  the  smaller  the  standard  error, 
the  better  the  estimating  equation.  The  F-ratio  is  defined 
as 


P _ Regression  mean  square  . 

Residual  mean  square  ’ 1 “ 

This  F-statistic  is  used  to  test  whether  the  incremental 
improvement  associated  with  the  addition  of  a variable  is 
significant.  Thus  the  larger  the  F-value  the  better.  The 
last  column  shows  the  coefficient  of  variation  which  relates 
the  standard  error  tSE) , to  the  mean  of  the  dependent 
variables  Y's,  or 
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This  value  is  used  in  comparing  one  standard  error  with 
another;  a lower  CV  value  is  better. 

B.  SELECTION  OF  THE  BEST  EQUATION 

Looking  at  Table  VI  , the  largest  measure  of  dispersion 
(R“)  is  0.98.  The  smallest  standard  error  is  0.083,  the 
highest  F-ratio  is  12.24  and  the  smallest  Coefficient  of 
Variation  (CV)  is  0.081.  Fortunately,  all  of  these  best 
values  for  measures  of  fit  occur  when  we  linearly  combine  the 
reciprocals  of  the  data.  Here  the  Coefficient  of  Variation 
(CV)  is  8.1%,  which  tells  us  the  estimating  equation  is 
fitted  very  well  in  this  case. 

The  coef f icients J that  provided  the  good  fitting  function 

were 

-11.62 
-74 . 15 
27.93 
-622.54 
271  .94 

Therefore  the  best  estimating  relationship  developed  amcnu 
those  forms  investigated  is 


EFFECTIVENESS 


11.62 


74.15  27.93  b22.54 

ARMOR  * SILHOUETTE  " SFEEr 
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See  Appendix 


B. 


The  closeness  of  the  fit  of  this  function  to  the  total 
effectiveness  values  furnished  by  the  judges  is  shown, 
for  individual  points,  in  Table  VII. 

Table  VII.  JUDGE  EFFECTIVENESS  VS  FUNCTION  EFFECTIVENESS 


TANK 

JUDGE 

EFFECTIVENESS 

FUNCTION 
EFFECT IVNESS 

DEVIATION 

PERCENT 

DEVIATION 

LEOPARD  1 

1.44 

1.46665249 

-0.266524897 

1.85% 

M60A1 

1.22 

1.165313536 

0.0546814318 

4 .48% 

T-62 

1.02 

0.9953316577 

0.0246683423 

2.42% 

M4  8A5 

0.89 

0 .9391371068 

-0 . 0491371068 

5.52% 

CENTURION 
(Mk.  13) 

0.78 

0.8004486225 

-0.0104486225 

1.32% 

AMX-30 

0.78 

0.773111555 

0.0068884449 

0.88% 

We  can  see  from  Table  VII  that  the  percent  deviations  of 
effectiveness  for  the  six  weapon  systems  are  all  less  than 
5.52%.  This  suggests  that  the  estimating  function  of 
reciprocals  of  the  data  fits  quite  well  the  data  upon  which 
it  was  developed. 

Let's  look  at  the  results  from  another  point  of  view. 

A common  practice  in  attempting  to  evaluate  the  effectiveness 
of  a weapon  system  is  to  use  a simple  linear  combination  of 
the  characteristic  values,  with  coefficients  determined  by 
any  of  several  rather  arbitrary  ways.  One  approach  which  is 
rarely  undertaken  is  to  do  as  was  done  in  this  paper,  using  a 
least  square  fit  with  effectiveness  values  obtained  from 
experts'  judgement. 
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Sj  and  which  will  make 


Figure  4 shows  the  scatter  plot  of  judge  effectiveness 
vs  function  effectiveness  estimated  from  a simple  linear 
function  of  the  characteristics , and  Figure  5 shows  the 
scatter  plot  of  judge  effectiveness  vs  function  effective- 
ness estimated  by  taking  the  reciprocal  of  the  characteristics. 

It  can  be  seen  at  a glance  that  the  function  effective- 
ness estimated  by  taking  reciprocal  of  the  characteristics 
is  much  closer  ibetter)  to  the  judge  effectiveness  than 
function  effectiveness  estimated  by  just  taking  a linear 
combination  of  the  characteristics . 

One  possible  reason  why  an  estimating  function  using 
reciprocals  of  the  characteristics  is  better  fitted  than 
the  common  procedure  of  evaluating  the  effectiveness  of 
a weapon  system  using  a simple  linear  combination  of  the 
characteristics , is  that  a property  like  system  effective- 
ness may  possess  diminishing  marginal  returns  with  respect 
to  increasing  character istic  values.  In  the  best  fitting 
equation  (reciprocals)  the  partial  derivative  of  effective- 
ness with  respect  to  a character istic  value  (with  a negative 
coefficient)  was  then  the  reciprocal  of  the  square  of  the 
characteristic  value,  hence  diminishing  marginal  returns. 

We  have  now  developed  a functional  relationship  between 
the  character istics  and  the  effectiveness  of  tank  systems 
as  intended  at  the  beginning  of  the  study,  and  found  that 
the  best  estimating  equation  among  those  considered  occurred 
when  a linear  combination  of  a reciprocal  of  the  character- 


istics was  used. 
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The  functional  relationship  developed  in  this  manner 
could  at  least  assist  military  planners  in  two  ways: 

(1)  in  assessing  the  impact  on  effectiveness  of  a modifica- 
tion of  characteristics  of  an  existing  tank,  and  (2)  in 
evaluating  the  effectiveness  of  a new  (and  perhaps  unbuilt) 
tank . 

We  will  summarize  the  results  of  our  study  and  propose 
some  recommendations  for  further  study  m the  next  chapter. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 

In  this  chapter  we  will  give  some  conclusions  from  the 
study,  and  some  suggestions  for  further  study. 

A.  CONCLUSIONS 

The  principal  purpose  of  this  study  was  to  determine 
whether  we  could  compute  the  overall  effectiveness  of  a 
weapon  system  from  its  characteristics,  to  establish  the 
existence  of  functions  which  could  be  used  to  relate  system 
characteristics  and  effectiveness,  and  to  identify  the  best 
estimating  relationship.  To  do  this  we  proposed  a procedure, 
sent  appropriate  questionnaires,  and  computed  the  overall 
effectiveness  values  for  tank  weapon  systems  by  using  the 
constant  sum  scaling  method.  Then,  using  multiple  regression, 
we  found  functional  relationships  as  in  Table  VI  and 
evaluating  these  results  we  finally  found  that  the  best 
estimating  equation  occurred  when  we  took  the  reciprocal, 
i.e.  , 


m 


(Effectiveness)  = a + ; b;( 

j=l 


1_ 

j 'X  . . 


where 


a = -11.6119  is  an  intercept, 


bj  = (-74.149,29.926,-622.536,291.944),  and 

are  characteristics  (Table  I) . 
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This  is  quite  an  interesting  result  because  most  people 
use  a linear  function  in  general. 

This  approach  is  felt  to  have  merit  as  a way  of  finding 
an  overall  MOE  because  it  is  based  on  the  opinion  of  many 
experts.  A conspicuous  limitation  in  the  tank  example  used 
here  is  that  this  study  did  not  include  tank  characteristics 
relating  to  fire  power,  primarily  because  of  the  difficulty 
of  data  collection. 

The  scaling  for  valuing  system  effectiveness  is,  of 
course,  independent  of  the  number  of  characteristics  or 
presence  of  data  on  those  characteristics.  This  provides, 
however,  effectiveness  values  only  for  the  instances  listed 
in  the  questionnaire.  Development  of  the  functional  rela- 
tionships between  characteristics  and  effectiveness,  unfor- 
tunately, requires  more  data  points  (instances  on  the  ques- 
tionnaire) than  character istics  if  the  function-finding 
approach  used  here  is  employed. 

B.  RECOMMENDATIONS  FOR  FURTHER  STUDY 

We  have  shown  how  to  develop  a model  by  which  one  can 
compute  the  system  effectiveness  directly  from  its  charac- 
teristics. However  we  feel  that  we  don't  know  how  good  our 
model  is.  To  test  our  model,  we  suggest  the  following 
method  which  could  be  part  of  a further  study. 

(1)  Select  instances  A.B.C.D.E  (or  any  number  of 
instances ) 

(2)  Send  questionnaires  on  A.B.C.D.  (excluding  instance 


appendix  a 
QUESTIONNAIRE 


PROBLEM  DESCRIPTION 


A fundamental  problem  in  Operations  Research  is  that  of  satisfactorily 
combining  various  MOEs  for  a system  into  a single  measure  of  overall 
effectiveness.  A variety  of  methods  have  been  used  out  of  necessity,  with 
the  weighted  sum  of  MOEs  being  perhaps  the  most  frequent  expedient.  However, 
both  the  values  of  the  weights  and  the  notion  of  adding  the  MOEs  are  usually 
difficult  to  justify,  and  this  approach  often  yields  results  which  differ 
sharply  from  expert  judgment. 

The  premise  of  our  study  at  NPS  is  that  individuals  with  expertise 
regarding  the  phenomenon  (in  our  case,  tanks)  possess  judgmental  ways  of 
combining  MOEs  which  are  far  superior  to  the  analyst's  simple  weighted 
average.  The  difficulty  is  that  the  expert  usually  cannot  tell  us  how 
he  knows,  say,  that  on  the  basis  of  given  data  Tank  A is  superior  to  Tank  B. 

Our  investigation  seeks  a procedure  to  approximate  the  way  experts 
put  MOEs  together  to  form  a single  measure  of  effectiveness.  This  approach 
crosses  disciplinary  lines  in  that  we  use  scaling  techniques  from  the 
behavioral  sciences  together  with  nonlinear  regression  methods  from  statistics. 

The  heart  of  the  approach,  of  course,  is  data  from  knowledgeable  people, 
obtained  through  a specialized  questionnaire.  The  research  is  still  in 
its  infancy,  and  the  current  effort  seeks  a small  amount  of  data  about 
a reduced  set  of  MOEs,  for  use  in  development  and  assessment  of  this 
approach. 

Degree  Candidate:  H.B.  Kim,  Lt,  Col.,  Korean  Army 

Adviser:  G.F.  Lindsay,  Assoc.  Prof,  of  Operations  Research,  Naval  Postgraduate 

School . 

14  October,  1978 


EFFECTIVENESS  OF  A TANK  WEAPONS  SYSTEM 

A study  is  being  made  of  various  measures  of  effectiveness  applicable 
to  a tank,  and  how  they  relate  to  overall  effectiveness.  Judgments  reflect- 
ing your  experience  and  expertise  are  solicited. 

Both  parts  of  this  questionnaire  have  purposefully  been  kept  short,  and 
we  ask  only  two  or  three  minutes  of  your  time. 

A . Importance  of  Tank  Effectiveness  Factors. 

Please  split  100  points  within  each  pair  listed  below,  assigning  a higher 
number  to  a factor  you  think  is  more  important.  For  example,  if  you  think 
that  fire  power  is  much  more  important  than  armor,  you  might  split  the  100 
points  as  follows: 

Fire  Power  AO  Armor  10 

Or,  if  you  thought  them  to  be  equal  in  importance,  you  would  write: 


Fire  Power  50 

Omit  pairs  you  feel  unable  to  rate 


1. 

Range , 

Road 

Fire  Power 

2, 

Range , 

Road 

t 

HP  ton 

1. 

Range , 

Road 

t 

Armor 

4. 

Range  , 

Road 

f 

Speed 

5. 

Range , 

Road 

/ 

Si lhouet te 

6 . 

Speed 

t 

Fire  Power 

7. 

Speed 

t 

HP  ton 

A. 

Speed 

t 

Armor 

0. 

Speed 

1 

S i lhouet te 

10. 

Fire  Powe  r 

HP  ton 

11. 

Fire  Power 

Armor 

12. 

Fire  Power 

Si lhouet  te 

13. 

HP  ton 

Armor 

14. 

HP  ton 

S il houette 

15. 

Armor 

Si  1 houette 

Armor  50 
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FACTORS 

M48AJT 

— 

M60A1 

AMX-30 

, 

LEOPARD  1 T-62 

1 

CENTURION 
( Mk  . 1 3 ) 

RANGE , ROAD 
(km) 

482 

500 

650 

1 

600  1 500 

1 

190 

FIRE  POWER 
(main  qun) 

105 

105 

105 

105 

us 

105 

SPEED 
(km/ hr) 

48.2 

48.3 

65 

65 

50 

34.6 

HP/ ton 

15.9 

15.3 

19.4 

20.7 

19.2 

12.5 

ARMOR 

(mm  on  nose) 

110 

110 

40 

'0 

100 

118 

SI  LHOUETTE 
(heirht  in  m) 

3.09 

3.26 

2.35 

2.64 

2.4 

1 

3.01  ! 

Some  characteristics  of  six  current  tanks  are  shown  above.  Please 
split  100  points  within  each  pair  listed  below,  in  terms  of  the  overall 
effectiveness  of  the  tank  weapons  system. 


i. 

M48A5 

M60A1 

2 

M48A5 

, AMX-30 

3. 

M48A5 

, LEOPARD  1 

4. 

M48A5 

T-62 

5 . 

M48A5 

, CENTURION 

6. 

T-62 

M60A1 

7. 

T-62 

AMX-30 

8. 

T-62 

, LEOPARD  1 

9. 

T-62 

, CENTURION 

10. 

AMX-30 

, M60A1 

11. 

AMX-30 

LEOPARD  1 

12. 

AMX-30 

, CENTURION 

13. 

CENTURION 

, M60A1 

14. 

CENTURION 

, LEOPARD  1 

15. 

LEOPARD  1 

, M60A1 

Thank  you  very  much  for  your  time  and  cooperation. 


!l 


i 
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APPENDIX  B 


MULTIPLE  REGRESSION  OUTPUT  ON  APL 


* Z<-Y  REGRESS  XJNlN.C.XPXINVJXPYSBETAiRSSJTSSiSESESSSWIDiDEP 
C13  X«-  ( 2t  ( p X ) * 1 ) p X 

C23  XHO.  1-6INTERCEPTU1  .X 
C33  XPXINV«-a<*X)+ . xX 
C 4 3 BETA«-XPXINV+.  xXPYMNX)  F.xY 
C51  RSS+-  ( <NBETA>+.  xXPY  )-€♦■<  ( f/Y  >*2  > -N<-p  .Y 
1163  ESS«-<  TSS<-i(NY)  + .xY)-C>-RSS 
C73  S2<- . ESS-.  ( N~  1 !-Kh  p » BETA  ) - a INTERCEPT 

C8  3 CR 

L"9]  1 ANQVA 1 

C 1 0 3 Cil«-  1 SOURCE  » DF  « SUM  SQUARES. MEAN  SQUARE  » F-RA  T TO  • 

C 1 1 3 1 1 

C12J  ’ OREGRESSIONCl.  I4.BE16. 4 • FMT(K)  . ( ,RSS)  . < .RSS-K  ) . < .RSS^K  >4-32 
C 133  CH«-'' 

C 1 4 3 1 n RESIDUALO. I4.BE16.4  ' FMT  ( (.  N- 1 > -M  * < * ESS  ) * S2 » Q 

C 15  3 'OTOTAL  !3.I4.  BE16. 4'  FMT  ( N- 1 ) » ( . ISS  ) . 0 . 0 

C 1 6 3 1 ' 

C 1 7 3 'R  SQUARE:  'J.RSS-TSS 

C 1i3  3 'STD  ERROR:  'f.S2*0.5 

C 1 9 3 UU«-‘ COEFFICIENTS.  T STATISTICS' 

C203  ' F 1 5 . 4 ' FMT*t 2. p .BETA) p < .BETA) .( .BETA ) -<1  1 NV<-S2  x XPX  INV  ) *0  , 


C 21 3 'DO  YOU  WANT  A PRINTOUT  OF  THE  VAR  I ANCE-COVAR  I ANCE  MATRIX?' 

C22  3 -»A1  < l ' Y ' *1  tO 

C233  ' VARIANCE-COVARIANCE  MATRIX:  ',CU«-'* 

11243  ' E12 . 4 ' FMT  V 

C25  3 Alt  ‘ IiURBIN-UATSON:  ' i ( P/(  < 11  »C)-C  ~U  .C)  )*2>-  + /<  .C*-Y-Xf  . x BETA  ' *: 
C2&3  Z *“*9  ( 2 . N ) p ( ,Xt«x  BETA  ) . . C 

C273  Bit' DO  YOU  WANT  TO  FORECAST  A VALUE  FOR  Y ? ' 

C283  aCI x 1 ' Y ' *1 10 

C29  3 'ENTER  X VECTOR  < 1 S fs.  ? ' VALUES)' 

C303  'FORECAST  OF  Y VALUE:  * it.  C<-  ( 1 -4  INTERCEPT  > 1 1 .□>  P . x BETA 
C31 3 'VARIANCE  OF  FORECAST  ERROR:  ' 5S2xlfCf. x XPX  INV+ . < NC 
C 32  3 aBI 

C333  Cll'OO  YOU  WANT  TO  SCAT  RESIDUALS  VS.  PREDICTED  Y" 

C 3 4 3 aOx i ' N ' *1 tQ 

C35  3 DEP*-0.5xuiD«-L/70.  < T/<  < 0. 75xN)  .30)  ) 

1136  3 SCAT  Z 
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>rtiMGE  OF  X : 0 . * 1 . .i 

MNGE  or  V : "0 .05  0.0 


ANOVA 


SOURCE  PE 
REGRESSION  •* 
RESIDUAL  1 
total  5 


SUM  SUUARFS 
3 . OS  1 *E~  t 
3.8565E”2 
3. 43’3E"1 


HE AN  SOU ARE 

’ . .*.29  or  *2 

3.85a5E*2 


F-KAT  10 

t .9 '..'2100 


R square:  0.88 ’8040559 

STP  ERROR:  0.19a380’9a2 

COEFFICIENTS  T STATISTICS 
*4.6385  *1.0011 


l .5055 
*’. 2268 
9 . *399 

*11. 0A4 ’ 

~ Too  UJANl  A LK  IN  four 


2.3534 
t . 4 '8a 
l . ’5a3 
1 . 500s 

E THE  OARIANCE-COVARTANCE 


Y 

VARIANCE -COVARIANCE  MATRIX! 


2.14 ’OE  t 1 
*9. 43*3E*2 


*9. 43*3E*2 
4 . 0”24E-1 


~ t . t a 1 5E  ft 
*2.5 ’O’EOO 


9, OO'IEOO 
3 . I O.V-'E  00 


” l • 1 A 1 5E  A I “2.570’E00  2.3888EM  *2.A094EM 

9 . 00  ’3E00  3 . 1 03«E00  * 2 . 609  4E  M 3 . 0 1 2aE  f 1 

*1.5d29EM  *3.9ft44E00  3.5500EE1  *4.0l9aEU 

miRBIN-UATSON:  2. 08051  ■'41  4 
PO  YOU  WANT  TO  FORECAST  A VALUE  FOR  Y ' 

N 

DO  YOU  WANT  TO  SCAT  RESIDUALS  VS . PREDICTED  >' 

Y 

RANGE  OF  X : 0 . a l . a 
RANGE  OF  Y : *0.15  0.15 


• 

«• 

4-  — ----^- 

1 . 4550 '23a  ’ 

*0.0150  '23a '29 

l . 093099  258 

0. 126900 '41  A 

0.9fl400-0802 

0 . 03599^1?  '5 

l .034541009 

*0. 1445410085 

0.  '9.M  936605 

-0.00a  19.36*0469 

0 . 777084625 

0 . 0029 1 33-5008 

MATS  1 X ' 


* 1 . 5A29F f 1 
' 1 . 9 a 4 4 E 0 0 
3.5500EF t 
‘4.01 9 At ♦ 1 
5 . 4 3 ’ 4 E f 1 


M 

» I 

'1 


( *Y  ' REGRESS  ( *X ) 


4N0VA 


SOURCE 

REGRESSION 

RESIDUAL 

TOTAL 


SUM  SQUARES 

2.  6622E"! 

3 . "504E"2 
3 . 0373E" 1 


MF AN  SQUARE 
6.6S53E”2 
3.  7504£"2 


F -RAT  10 
, 7-46E00 


R SQUARE:  0.8765190! 75 

std  error:  o.  1936598687 


COEFFICIENTS 

"6.0277 

1.3669 

“5.9679 

0.1447 

~9.lt 04 

Yiiu  want  a rr: 


STATISTICS 
"1 .3192 


THE  VAR [ANCE -COVARIANCE  MATRIX’ 


VARIANCE-COVARIANCE  MATRIX: 


2.08~,9EE1 
"9. l?"6E~2 
"1 . t 295E  f 1 
0 . "394E00 
"1 .5198EFI 
durbin-uatson: 

DO  YOU  WANT  TO 
N 

DO  YOU  WANT  TO 
Y 

RANGE  OF  x:  "0 
RANGE  OF  YI  "0 


■9,i;’76E~2  "1.1293E41  8."594E00  “I.S198E+1 

3.9M98E"!  ”2 . SOOOEOO  3.01S5E00  "3.8553E00 

2.5000E00  2.3231EE1  “2.53"6Etl  3.4523EM 

3 . 0185E00  "2.3376EM  2.929"Efl  “3.9090FEI 

‘3 . 8553E00  3.4S23E+1  “3.9090EE1  5.28’8E+l 

2.080517414 

FORECAST  A VALUE  FOR  Y 7 

SCAT  RESIDUALS  VS.  PREDICTED  Y ? 


0«3  "*^5066 4 

"0.0148635341" 

0. 073 "083 "3 12 

0.  1251 424856 

"0.0 1569139-46 

0 . 03549422  4 "5 

0.02600452402 

"0. 1423383403 

"0. 2296144885 

“0.006107845058 

~0. 2513343684 

0 . 002S"3009 1 2" 
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SOURCE  OP  SUM  SQUARES  MEAN  SQUARE  F-RATTO 

REGRESSION  4 3.3*85E-1  8.42I3E~2  1.2240E+I 

RESIDUAL  l 6.8800E-3  6 . 8800E~3 

TOTAL  5 3.4373E-! 


I ANCE-CO  VARIANCE  MATRIX1? 

Y 

variance-covariance  matrix: 

1.1415E1  2.7450E1  '2.2374E1  4.3981E2  '2.1281E2 

2 . 7450E1  1.4130E2  '5.7579E1  1 . 2390E3  "5.**55E2 

~2 . 2374E 1 "5.7570E1  4.4I87E1  “8.7077E2  4.1895E2 

4.3981E2  1.2390E3  '8.7077E2  1.7803E4  "8.4757E3 

“2.1291E2  "5 . 6o55E2  4.I8C5E2  “8.4757E3  4.0*15E3 

durb  in-watson:  2. 145430074 

DO  YOU  WANT  to  FORECAST  A VALUE  FOR  Y? 

N 

DO  YOU  WANT  TO  SCAT  RESIDUALS  VS,  PREDICTED  Y? 

Y 


RANGE  OF  X:  0.6  t • 6 
RANGE  OF  Y.*  ~0.0*  0.0* 
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I 


1-Y'  REGRESS  1-X.’ 


ANOOA 


SOURCE 

REGRESSION 

RESIDUAL 

TOTAL 


DF 

4 

1 


SUM  SQUARES 
2.751 1E”1 
1 .2058E”2 
2.871 7E~ 1 


MEAN  SQUARE 
A . 8777E ”2 
1 .2058E”2 


F-RATIO 
5. 7039E00 


R SQUARE:  0.9580105484 
STD  ERROR!  0.1098086393 

COEFFICIENTS  T STATISTICS 


9, 
65, 
“20, 
471  , 


9859 

7943 

6599 

4091 


'200.6539 


.2326 
. 181 
.3477 
.6688 
.3783 


DO  YOU  WANT  A PRINTOUT  OF  THE  OARI ANCE-CQOAR IANCE  MATRIX? 


oariance-cooariance  matrix: 


0005E1 

3109E1 

9213E1 

7082E2 

7296E2 


, 8109E1 
, 4764E2 
, 009 1 E2 
, 1714E3 
'9.9294E2 
durbin-uatson:  2.145430874 

DO  YOU  WANT  TO  FORECAST  A VALUE  FOR  Y' 
N 


9213E1 

0091E2 

7442E1 

5261E3 

3424E2 


. 7082E2 
, 1714E3 
, 526 1 E3 
, 1201E4 


■3. 

~<3  . 


'1 . 4854E4 


7296E 
9294E 
3424E 
4854E4 
1 182E3 


DO  YOU  WANT  TO  SCAT  RESIDUALS  OS. 
Y 

0.6  1.4 

“0.08  0.08 


PREDICTED  Y' 


RANGE  OF  X : 
RANGE  OF  Y : 


0.6591603144 

0.03528413003 

0 . 8920*26 197 

”0.07239048852 

1.013049556 

”0 . 03265739021 

1.05854492 

0.06505058572 

1 . 251990285 

0.01383249969 

1 .29117061 

“0 . 00° 1 1 932772 

ci  n ri 


Y REGRESS  X*.' 


1 

SOURCE  DF 
REGRESSION  4 

RESIDUAL  t 

TOTAL  5 

ANOVA 

SUM  SQUARES 

2 . 8793E” 1 

5 . 5799E”2 

3 . 4373E”  1 

ft  SOIJrtftE  ♦ 0*837 

6671578 

STD  ERROR:  0.236 

2185618 

COEFFICIENTS 

T STATISTICS 

9.6981 

0.7228 

0.3986 

t .8048 

”7.0421 

”1.0655 

2.3585 

1 .3374 

”4 . 2137 

”1 . 0^44 

00  YOU  WANT  A PRINTOUT  OF  H 

Y 

VARIANCE-COVARIANCE  MATRIX: 


MEAN  SQUARE 
7. 1984E”2 
5 . 5799E”2 


E-RATTO 
1 . 2900EOO 


“iR I ANCE-COVAR I ANCE  MATRIX’ 


'8  . 6857E+ 1 


1 . 8003E  f 2 2 . 1507E00  "8.6857E+1  2.1746 

2.1507E00  4 . 3783E”2  “1.22S1E00  3.5430 

‘a.oa57Efl  "1 . 2281E00  4.3683E+1  ”1.1273 

2.1746E+1  3.5430E”!  “t.l273E+l  3.1101 

5.0610E+1  ”7 , 43 1 6E” 1 2.5479E+1  ”6.8470 

durbin-uatson:  2.033119534 

DO  YOU  WANT  TO  FORECAST  A VALUE  FOR  Y 
N 

DO  YOU  WANT  TO  SCAT  RESIDUALS  VS.  PREDICTED  Y 


'1 . 2281E00 
4 . 3633E+1 
’1  . 1273E+1 
2 . 5479E+ 1 


2 . 1746EP1 
3 . 5430E” 1 

3. 1 101E00 
6 . 8470EOO 


‘5.0610E+1 
‘7.431 &E— 1 
2 . 5479E+  l 
‘6 . 84“’OEOO 
1 . 5381E+1 


RANGE  OF  X:  0.6  1.6 

RANGE  OF  Y:  “0.2  0.15 
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y repress  <1  x».5> 


t 


ANOVA 


SOURCE 

DF 

SUM  SQUARES 

MEAN  SQUARE 

F-RA  T I u 

K'f  GRESS  [ON 

4 

3 . 2224E~t 

8 . 0559E" 2 

3. 7474E00 

RESIDUAL 

l 

2. 1497E"2 

2. 1497E"2 

TOTAL 

5 

3. 43  73E~1 

R SQUARE  : 0 . 93 7459535  7 

SID  ERROR:  0. t 466193789 


COEFFICIENTS 
"10. 7584 
"21 . 7109 
28. 71*2 
"155.6577 
111.261 4 


I STATISTICS 
"1 . 7598 
“3.367 
2.2021 
“2.5165 
2 . 2382 


00  YOU  WANT  A PRINTOUT  OF  THE  VARIANCE 
Y 


COVARIANCE 


VAR IANCE -COVARIANCE  A A TR I X I 


1 . 1362E2 
5.022 IE 1 
"l . 581 7E2 
6 . 3622E2 
"5.21 1 8E2 


5.0221E1 
4. 1580E1 
"6.6284E1 
3.3699E2 
"2.5875E2 


“ 1 . 33 1 7E2 
6 . 6284C  t 
1 . 7005E2 
"7.8844E2 
6.3979E2 


A . 5627E2 
3.3699E2 
"7.8844E2 
3.8260E3 
"3.0602E3 


OIJRBIN- WATSONJ  2.  122115702 

DO  YOU  WANT  TO  FORECAST  A VALUE  FOR  Y 7 

N 

DO  YOU  WANT  TO  SCAT  RESIDUALS  VS.  PREDICTED  Y 5 
Y 

RANGE  OF  X • 0 . A 1.6 
RANGE  OF  Y : "0.1  0.1 


MATRIX  ’ 


“5.2U8E2 
‘2.5875E2 
A.3979E2 
“3.0A02E3 
2.4  7 1 1 E3 


a 


'l  . 46  7833212 

"0.02783321 t "2 

L . 123 151  At 

0.0968483904-5 

0.9852532576 

0.03474674236 

0.9897234947 

"0.09972349474 

0 . 80 15763548 

"0.01 157635483 

0 . 7724*20  71,‘> 

0.007537928422 

5 1 


1 


ANOVA 


SOURCE  OF  SUM  SQUARES  MEAN  SQUARE  F-RATIO 

REGRESSION  4 &.6737E~2  1.6684E"2  1.2494F00 

RESIDUAL  I t.3354E“2  1 . 3354E"2 

TOTAL  5 8 . 009 1 E~2 


|R  SQUARE.’  0. 83326 7c 

| 

STD  ERROR  2 0.1155581755  ! 

COEFFICIENTS  T 

STATISTICS 

4.6632 

0.7104 

0.1884 

1 . 7438 

"3.1607 

"0.9776 

1.076 

1.2472 

"1 .8862 

"0.9831 

DO  YOU  WANT  A PRINTOUT  OF  “HE  VAR  I ANCE -COVAR I ANCE  MA  TR I X? 

Y 

VARIANCE-COVARIANCE  MATRIX ! 

4 . 3085E+1  5.1469E"1  "2.0786EP1  5.2041E00  "1.2112E+1 

5 . 1 469E~1  1.1675E"2  ~2.9389E~1  8.4790E"2  ~l.?785E"t 

”2 . 0786E+ 1 ~2.9389E“t  1.0454E+1  "2.6978E00  6.0975E00 

5.2041E00  8 . 4790E“"2  "2.6978E00  7.4430E“1  “1.6386E00 

"1.21 12E+1  ~ 1 . 7785E- 1 6.0975E00  "1.6386E00  3.6810E00 

DURBIN-UATSONI  2.038119584 
DO  YOU  WANT  TO  FORECAST  A VALUE  FOR  Y? 

N 

DO  YOU  WANT  TO  SCAT  RESIDUALS  VS.  PREDICTED  Y^ 

Y 

RANGE  OF  X:  0.85  1.2 

RANGE  OF  YJ  -0.1  0.1 
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APPENDIX  C 


COMPUTATION  OF  A RATIO  SCALE  OF  THE 
IMPORTANCE  OF  SIX  MAJOR  FACTORS  OF 
A TANK  WEAPON  SYSTEM,  USING  THE 
CONSTANT  SUM  SCALING  METHOD 

As  a supplementary  study,  we  computed  a ratio  scale 
of  the  importance  of  six  major  factors  of  a tank  weapon 
system.  The  factors  were: 

Road  Range , 

Fire  Power, 

Speed, 

Hp/ton , 

Armor , and 
Silhouette . 

As  we  did  in  Chapters  II  and  III  for  system  effectiveness, 
we  followed  the  same  procedure,  i.e.,  we  selected  the 
major  factors,  collected  data,  selected  the  constant  sum 
scaling  method,  sent  questionnaires  together  with  the  ques- 
tionnaires for  tank  weapon  system  (Appendix  A) , and  computed 
the  ratio  scale  values  about  the  importance  of  each  of  the 
six  major  factors. 

The  a^s  array  was  computed  from  the  50  judges  responses 
and  we  computed  the  VT  ^ array  from  this  a~  array  by  taking 
the  ratio  of  the  average  points  awarded  to  instance  j (when 
compared  to  instance  i)  to  the  average  points  awarded  to 
instance  i (when  compared  to  instance  j).  After  constructing 


the  Wij  array  we  used  equation  (11),  i.e., 

n 

Sj  - I l*ij>1/n  , j = 1,2,  . . . , n 
i-1 


to  get  the  ratio  scale  values  of  factors.  The  ratio  scale 
values  computed  are  shown  in  Table  C,  and  represented 
graphically  in  Figure  a. 


Table  A.  a_  ARRAY  COMPUTED  FROM  JUDGES  RESPONSES 


RANGE, 

ROAD 

fire 

POWER 

SPEED 

HP/TON 

ARMOR 

SILHOUETTE 

RANGE,  ROAD 

50 

75 

58.7 

59.8 

66 

63.3 

FIRE  POWER 

25 

50 

32.5 

30.7 

38.3 

36.6 

SPEED 

41.3 

69.5 

50 

43.1 

58.7 

55.6 

HP/TCN 

42.2 

63.3 

50.9 

50 

59.3 

54.7 

•ARMOR 

34 

61.9 

41.3 

42.7 

50 

55.5 

SIIBOUEITE 

36.4 

63.4 

44.4 

45.3 

54.5 

50 

57 


1 


Table  B Wi;j  ARRAY 


RANGE, 

FIRE 

ROAD 

POWER 

SPEED 

HP,  TCN 

ARMOR 

SILHOUETTE 

RANGE,  ROAD 

1 

3 

1.421 

1.369 

1.941 

1.949 

FIRE  POWER 

.333 

1 

.481 

.443 

.621 

.599 

SPEED 

.703 

2.099 

1 

.965 

1.421 

1.252 

HP,  TON 

.730 

2.259 

1.037 

1 

1.342 

1.209 

ARMOR 

.515 

1.611 

.703 

.745 

1 

.335 

SILHOUETTE 

.592 

1.932 

.798 

.828 

1.198 

1 

so  = 

. 61 

1.84 

.86 

.84 

1.18 

1.04 

H 


Table  C RATIO  SCALE  OF  SIX 
FACTORS  OF  TANK  WPN 


FACTORS 
FIRE  POWER 
ARMOR 
SILHOUETTE 
SPEED 
HP /TON 
RANGE , ROAD 


MAJOR 
SYS. 

RATIO  SCALE 
1.84 
1.13 
1.04 
0 . 86 
0.84 
0.61 
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